Surgical navigation system

ABSTRACT

A surgical navigation system is provided. The surgical navigation system comprises a camera system comprising at least one camera. The camera system is capable of capturing images of light that is filtered through at least three different polarization angles. The surgical navigation system further comprises a processor having access to three dimensional image data of a surgical object. The processor is configured to determine first surface normals from the three dimensional image data of the surgical object. The processor is further configured to determine second surface normals from at least three images captured by the camera system of the surgical object under different polarization angles. The processor is configured to align the first surface normals with the second surface normals. Furthermore, a method for operating a surgical navigation system is provided.

RELATED APPLICATIONS

This application claims priority to European Patent Application No. 18176 829.2, filed Jun. 8, 2018, the content of which is incorporatedherein by reference in its entirety.

TECHNICAL FIELD

The present disclosure generally relates to a surgical navigationsystem. In particular, a surgical navigation system comprising a cameracapable of capturing images filtered through different polarizationangles is presented.

BACKGROUND

During surgery, a surgeon typically inserts an instrument into the bodyof a patient. Once inside the patient's body, the surgeon loses visionof the tip of the instrument. In order to help the surgeon navigate theinstrument in such a case, a surgical navigation system can be used thattracks the position and/or orientation of the instrument relative to thepatient and provides visual or acoustic guidance to the surgeon. One wayto achieve this is to calculate a position of a virtual instrument(e.g., a CAD data set of the instrument) relative to a virtual patient(e.g., a CT scan of the patient) and to display on a monitor where thecalculated position of the virtual instrument tip is located relative tothe virtual patient.

Navigation in regard to surgical objects, such as the patient or asurgical instrument, typically comprises two procedures: registrationand tracking. During registration, a spatial relationship betweendifferent coordinate systems (e.g., of the patient in an operating roomand a previously acquired CT scan of the patient) is determined. This iscommonly achieved by determining a transformation between the associatedcoordinate systems. After successful registration, a surgical object canbe tracked, wherein changes of the position and/or orientation of thesurgical object are determined (e.g., via a camera in the operatingroom) in one coordinate system and transformed into the other coordinatesystem.

It has recently been proposed to utilize surface-from-polarization, SFP,techniques for surgical applications. Such SFP techniques comprise acapturing of images under different polarization angles and areconstruction of surface normals from these captured images. Prior artin this regard is known from US 2016/0261844 A1 which disclosescapturing a first and a second set of images, wherein polarizationsurface normals are calculated based on the second set of images. Afirst depth map is calculated based on the first set of images and asecond depth map is calculated based on the first depth map andpolarization surface normals. This means that in order to calculate thesecond depth map, a first depth map needs to be acquired.

SUMMARY

There is a need for a surgical navigation system that improves theregistration and/or tracking for a surgical object.

According to one aspect, a surgical navigation system is provided. Thesurgical navigation system comprises a camera system comprising at leastone camera, wherein the camera system is capable of capturing images oflight that is filtered through at least three different polarizationangles. The surgical navigation system further comprises a processorhaving access to three dimensional image data of a surgical object. Theprocessor is configured to determine first surface normals from thethree dimensional image data of the surgical object. The processor isfurther configured to determine second surface normals of the surgicalobject from at least three images captured of the surgical object by thecamera system under different polarization angles. The processor is alsoconfigured to align the first surface normals with the second surfacenormals.

The camera system may comprise a single camera. The camera system mayalternatively comprise a plurality of cameras. The plurality of camerasmay be spatially spaced apart.

The light for the captured images may be filtered through an adjustableor non-adjustable polarizing filter. The camera system may comprise aplurality of cameras each with a polarizing filter, wherein thepolarizing filters have different polarization angles relative to eachother. The camera system may comprise optically active elements, such asliquid crystal cells. The optically active elements may be configured torotate a polarization angle of polarized light entering the camerasystem. The optically active elements may be electrically operable.

The surgical object may be a patient or a part thereof or a surgicalinstrument. The three dimensional image data of the patient may beacquired by computed tomography, CT, magnetic resonance imaging, MRI,positron emission tomography, PET, ultrasound or other scanningprocedures that acquires three dimensional patient data. The threedimensional data of the surgical instrument may be acquired in the formof a CAD data set, a three dimensional laser scan or dimensionalparameters that were entered by the user.

The processor may be configured to align the second surface normals withthe first surface normals to determine a transformation of coordinatesof the first surface normals in a first coordinate system into a secondcoordinate system of the second surface normals. The processor may beconfigured to align the second surface normals with the first surfacenormals to determine an inverse transformation of coordinates of thesecond surface normals in a second coordinate system into a firstcoordinate system of the first surface normals. The transformationand/or inverse transformation may be determined by a minimization of asum of differences between the first surface normals and the secondsurface normals. The transformation may also be determined by aminimization of a sum of differences between projections of the firstsurface normals and the second surface normals onto an image plane ofthe camera system. A difference may be calculated as the magnitude of avector difference between two surface normals.

The processor may be configured to differentiate if the surgical objectis of a first type or a second type based on a degree of polarizationdetermined for the second surface normals of the surgical object. Theprocessor may be configured to determine that the surgical object is ofthe first type if a degree of polarization determined for the secondsurface normals of the surgical object is lower than a first threshold.Additionally, or in the alternative, the processor may be configured todetermine that the surgical object is of the second type if a degree ofpolarization determined for the second surface normals of the surgicalobject is higher than a second threshold. The first type surgical objectmay be a patient and the second type surgical object may be a surgicalinstrument. The first threshold may be identical to the secondthreshold.

The processor may be configured to differentiate if a second surfacenormal of the second surface normals is pertaining to a surface orsurface region covered by liquid based on a degree of polarizationdetermined for the second surface normal of the second surface normals.The processor may be configured to determine that the second surfacenormal of the second surface normals is pertaining to a surface orsurface region covered by liquid if the degree of polarizationdetermined for the second surface normal of the second surface normalsis above a third threshold. The surface covering liquid may comprise atleast one of blood, water, disinfectant and pus.

The processor may be configured to differentiate if a second surfacenormal of the second surface normals is pertaining to a surface emittingor reflecting light based on a degree of polarization determined for thesecond surface normal of the second surface normals. The processor maybe configured to identify that the second surface normal of the secondsurface normals is pertaining to a light emitting surface if the degreeof polarization determined for the second surface normal of the secondnormals is below a fourth threshold. The processor may be configured toidentify that the second surface normal of the second surface normals ispertaining to a light reflecting surface if the degree of polarizationdetermined for the second surface normal of the second normals is abovea fifth threshold. The fourth threshold may be identical to the fifththreshold.

The processor may be configured to determine the second surface normalsfrom the at least three images by fitting a light intensity of pixels ofthe captured images to a sinusoid model that describes a magnitude of alight beam having a light beam polarization angle passing through apolarization filter. The processor may be configured to determine thelight polarization angle and a degree of polarization from the fittedlight intensity.

The processor may be configured to determine the second surface normalsfrom the at least three images by using a Fresnel equation. A dielectricconstant used in the Fresnel equation may be set to a value between 1.3and 1.6.

The processor may be configured to determine or have access to at leastone parameter of at least one camera of the camera system that isindicative of at least one of a spatial position and optical propertiesof the at least one camera of the camera system. The processor may beconfigured to determine or have access to at least one parameter of eachof multiple cameras of the camera system. The at least one parameter maybe indicative of a position and/or orientation of the at least onecamera relative to the surgical object and/or another camera. The atleast one parameter may be indicative of at least one of a distortion, afocal length, a lens arrangement and a sensor position relative to thelens arrangement.

The processor may be further configured to track a second type surgicalobject relative to a first type surgical object by calculating aposition and orientation of the second type surgical object relative tothe first type surgical object. The processor may be configured to trackthe second type surgical object and the first type surgical objectrelative to a camera coordinate system by calculating a position andorientation of the second type surgical object and the first typesurgical object relative to the camera coordinate system. The first typesurgical object may be a patient and the second type surgical object maybe a surgical instrument.

The three dimensional image data may be previously acquired from anunknown position and/or viewing angle. For example, the position and/orviewing angle of an imaging device that has acquired the threedimensional image data may be unknown. Specifically, a relationship(e.g., the transformation parameters) between the viewing angle of theimaging device and the viewing angle of the camera system may beunknown.

According to a second aspect, a method for operating a surgicalnavigation system is provided. The surgical navigation system comprisesa camera system comprising at least one camera, wherein the camerasystem is capable of capturing images of light that is filtered throughat least three different polarization angles. The processor further hasaccess to three dimensional image data of a surgical object. The methodcomprises determining first surface normals of the surgical object fromthe three dimensional image data. The method further comprisesdetermining second surface normals of the surgical object from at leastthree images of the surgical object captured by the camera system underdifferent polarization angles. The method comprises aligning the firstsurface normals with the second surface normals.

According to a third aspect, a non-transitory computer-readable mediumthat stores programming instructions for execution, that when executedon at least one processor, cause the at least one processor to carry outthe method explained above.

BRIEF DESCRIPTION OF THE DRAWINGS

Further details, advantages and aspects of the present disclosure willbecome apparent from the following embodiments taken in conjunction withthe drawings, wherein:

FIG. 1 shows a light sensor detecting a light beam that was reflectedoff the surface of an object;

FIG. 2 shows a light sensor detecting the intensity of a polarized orpartially polarized light beam after passing through a polarizationfilter;

FIG. 3 shows an embodiment of a surgical navigation system comprising acamera system with one camera;

FIG. 4 shows a flow diagram of a method embodiment of operating asurgical navigation system;

FIG. 5 shows an embodiment of an alignment procedure of first surfacenormals and second surface normals;

FIG. 6 shows an embodiment of a surgical navigation system comprising acamera system with a plurality of cameras;

FIG. 7A shows an embodiment of a surgical navigation system with arotatable polarizing filter in combination with a stereo camera;

FIG. 7B shows an embodiment of a stereo camera, wherein one of the twocameras comprises a rotatable polarizing filter;

FIG. 7C shows an embodiment of a stereo camera, wherein both camerascomprise a rotatable polarizing filter; and

FIG. 7D shows an embodiment of an apparatus for electromagnetic trackingin combination with a surgical navigation system.

DETAILED DESCRIPTION

In the following description, exemplary embodiments of a surgicalnavigation system and a method for operating a surgical navigationsystem will be explained with reference to the drawings. The same orsimilar reference numerals will be used to denote the same or similarstructural features.

The embodiments described below incorporate a technique commonlyreferred to as “Surface-From-Polarization”, or SFP. Before describingthe embodiments in greater detail, the physical background of SFP isdescribed first.

With reference to FIG. 1 it will be explained how a light beamreflecting of a surface is polarized and in what way the surface normalat the point of incidence is correlated to the polarization of the lightbeam. Using an exemplary setup with a light sensor shown in FIG. 2, itwill then be explained how to gather information from the reflectedlight beam that allows determining the surface normal. The principle isthen expanded to a plurality of light sensors in the form of a pixelarray, which allows generating a surface normal map.

FIG. 1 shows a light sensor 10 detecting a light beam 12 that wasreflected off the surface of an object 14. The light sensor 10 maybelong to any of the cameras or camera systems discussed in more detailbelow. The exemplary object shown in FIG. 1 is a sphere. The object 14may be any other object with a surface, in particular a surgical objectsuch as a patient or a surgical instrument. The surface normals of theobject 14 are to be determined to allow registering and tracking of thesurface.

When the light beam 12 is reflected off the surface, the path of thelight beam 12 is arranged in an incidence plane 18 that comprises asurface normal 20 at a point of incidence 22 of the light beam 12. Theincident and reflection angles θ of the light beam 12 are equal andformed between the light beam 12 and the surface normal 20 with thepoint of incidence 22 forming the vortex. The light sensor 10 has an(imaginary) image plane 16. The reflected light beam 12 is essentiallyperpendicularly oriented relative to the image plane 16. Since thereflected light beam 12 extends perpendicular to the image plane 16 ofthe light sensor 10, the reflection angle θ also defines a zenith angleθ relative to the light sensor 10. Therefore, when determining theorientation of the surface normal 20, one part of the orientation of thesurface normal 20 may be determined by calculating the azimuth angle θ.

The other part of the orientation of the surface normal 20 is determinedby an azimuth angle φ. The azimuth angle φ of the surface normal 20 iscorrelated with the polarization of the light beam 12. A reflected lightbeam is at least partially polarized in a direction perpendicular to theplane of incidence 18 and therefore also the surface normal 20.Consequently, the angle of polarization φ is also the azimuth angle φ ofthe surface normal 20 relative to the light sensor 10.

In the following, it will be explained how the azimuth angle φ and thezenith angle θ can be determined from detecting reflected light underdifferent polarization angles of a polarizing filter.

FIG. 2 shows the light sensor 10 detecting the intensity of thepolarized or partially polarized light beam 12 after passing through apolarization filter 26. The polarization filter 26 is orientedessentially in parallel to the image plane 16. The polarization filter26 is rotatable, e.g., by means of a motor 28, wherein a polarizationfilter angle β relative to a reference axis 29, which in FIG. 1 is avertical axis, can be adjusted by a rotation of the polarization filter26.

The light beam 12 in FIG. 2 is essentially perpendicularly orientedrelative to the polarization filter 26 and the image plane 16.Furthermore, the light beam 12 has a light beam polarization angle φbetween the reference axis 29 and the polarization angle of the lightbeam 12.

When the polarization of the light beam 12 and the polarization filter26 align (i.e. β≈φ), the light sensor 10 senses a maximum lightintensity I_(max). If the initial light beam 12 is (completely)polarized, the light beam 12 then passes the polarization filter 26essentially unattenuated. If the initial light beam 12 is only partiallypolarized, the attenuation of the polarization filter 26 is minimizedwhen β=φ.

In the case the polarization of the light beam 12 and the polarizationfilter 26 are aligned perpendicular to each other (i.e. β≈φ±90°, thelight sensor 10 senses a minimum intensity I_(min).

While rotating the polarization filter 26, the light sensor 12 detects alight intensity that falls between I_(min) and I_(max), wherein thedetected intensity I(β, φ) can be described by equation (1):

$\begin{matrix}{{I\left( {\beta,\varphi} \right)} = {\frac{I_{\max} + I_{\min}}{2} + {\frac{I_{\max} - I_{\min}}{2} \cdot {\cos\left( {{2\beta} - {2\varphi}} \right)}}}} & \left( {{Eq}.\mspace{11mu} 1} \right)\end{matrix}$

Equation (1) has three parameters: The maximum light intensity I_(max),the minimum light intensity I_(min) and the light beam polarizationangle φ. In order to identify these three parameters, the lightintensity is measured for three different polarization filter angles β₁,β₂ and β₃. The three polarization angles may for example be β₁=0°,β₂=45° and β₃=90°. Under these three polarization angles, the lightsensor detects three different intensities I₁, I₂ and I₃. The light beampolarization angle φ can be determined using equation (2) which is basedon equation (1):

$\begin{matrix}{{{\varphi = {{\frac{1}{2}{\arctan\left( \frac{I_{1} + I_{3} - {2I_{2}}}{I_{3} - I_{1}} \right)}} + {90{^\circ}}}},{{{if}\mspace{14mu} I_{3}} < {I_{1}{or}\mspace{14mu} I_{2}} < {I_{1}\mspace{14mu}{or}}}}{{\varphi = {{\frac{1}{2}{\arctan\left( \frac{I_{1} + I_{3} - {2I_{2}}}{I_{3} - I_{1}} \right)}} - {90{^\circ}}}},{{for}\mspace{14mu}{any}\mspace{14mu}{other}\mspace{14mu}{{case}.}}}} & \left( {{Eq}.\mspace{11mu} 2} \right)\end{matrix}$

Therefore, by sensing the intensity of the light beam 12 and applyingequation (2) to the obtained intensity values, one can calculate thelight beam polarization angle φ, which is also, as explained above, theazimuth angle φ of the surface normal 20 (see FIG. 1). It should benoted that equation (1) has the same solution for φ as well as φ±n.Consequently, the azimuth angle φ calculated by equation (2) has a180-degree angle ambiguity. This issue will be addressed further below.

The other angle that needs to be determined is the zenith angle θ. Forthis purpose, the following equation (3) is used, which describes thedegree of polarization ρ:

$\begin{matrix}{\rho = \frac{I_{3} - I_{1}}{{\left( {I_{3} + I_{1}} \right) \cdot \cos}\; 2\;\phi}} & \left( {{Eq}.\mspace{11mu} 3} \right)\end{matrix}$

Substituting the Fresnel Equations into equation (3) yields thefollowing equation (4) for the polarization ρ:

$\begin{matrix}{{\rho = \frac{\left( {n - \frac{1}{n}} \right)^{2}\sin^{2}\theta}{2 + {2n^{2}} - {\left( {n + \frac{1}{n}} \right)^{2}\sin^{2}\theta} + {4\cos\;\theta\sqrt{n^{2} - {\sin^{2}\theta}}}}},} & \left( {{Eq}.\mspace{11mu} 4} \right)\end{matrix}$wherein n denotes the refractive index and θ the zenith angle. Therefractive index n is usually not known. However, for most dielectricsthe refractive index can be set to a value between 1.3 and 1.6. With aset refractive index n (e.g., 1.4) and the degree of polarization knownfrom equation (3), the zenith angle θ can be estimated from equation (4)(e.g., in closed-form or through numerical optimization).

Equation (4) may be applied for diffusive reflection. In the case ofspecular reflection, equation (5) may be used

$\begin{matrix}{{\rho_{sepc} = \frac{2n\;\tan\;{\theta sin\theta}}{{\tan^{2}{\theta sin}^{2}\theta} + {n^{*}}^{2}}},} & \left( {{Eq}.\mspace{11mu} 5} \right)\end{matrix}$with |n*|²=n²(t+κ²), wherein κ is the attenuation index of the materialand the degree of specular polarization is ρ_(spec).

With the azimuth angle φ and the zenith angle θ of the surface normal 20calculated via the above equations (1) to (5), a vector {right arrowover (p)} of the surface normal 20 can be calculated by the followingequation (6):

$\begin{matrix}{\overset{\rightarrow}{p} = {\begin{pmatrix}p_{x} \\p_{y} \\p_{z}\end{pmatrix} = \begin{pmatrix}\begin{matrix}{{\cos\;\alpha}\;} & {\cos\;\theta} \\{\sin\;\alpha} & {\sin\;\theta}\end{matrix} \\{\cos\;\theta}\end{pmatrix}}} & \left( {{Eq}.\mspace{11mu} 6} \right)\end{matrix}$with α=φ or φ+π.

The above equations allow calculating the vector of a surface normalrelative to a single light sensor 10. Therefore, one light sensor 10 maybe assigned one surface normal 20. By employing a camera with an arrayof light sensors 10, a plurality of corresponding surface normals may bedetermined. Such an array of light sensors may be provided by a digitalcamera in form of a charge-couple device, complementarymetal-oxide-semiconductor or other image sensor with an array of lightsensors that allow recording a plurality of pixels.

FIG. 3 shows an embodiment of a surgical navigation system 30 comprisinga camera system 32 with one camera 34. The camera 34 comprises a pixelarray (e.g., a CCD array) that essentially comprises a plurality oflight sensors 10 as discussed above. The camera system 32 furthercomprises a rotatable polarization filter 26 which is configured topolarize light entering an objective 35 of the camera 34. By adjustingthe polarization angle of the polarization filter 26, the polarizationangle of light entering the camera 34 can be adjusted.

The surgical navigation system 30 further comprises a processor 36 thatis configured to receive image data of the camera system 32. Theprocessor 36 may receive the image data via a cable, wireless datatransfer or an external storage medium that receives the image data fromthe camera system 32.

The processor 36 is configured to control the polarization angle of thepolarization filter 26. Alternatively, the camera system 32 or thecamera 34 may be configured to control the polarization angle of thepolarization filter 26. In such a case, the camera system 32 or camera34 captures images under different polarization angles and sends theimage data to the processor 36 together with information that indicatesunder which polarization angle the images were captured.

The surgical navigation system 30 further comprises a storage medium 38.The storage medium 38 stores three dimensional image data of one or moresurgical objects. The surgical objects may comprise a patient 40 and asurgical instrument 42. Three dimensional image data of the patient 40may have been acquired by computed tomography, CT, magnetic resonanceimaging, MRI, positron emission tomography, PET, ultrasound or otherscanning procedures that acquire three dimensional data of the patient40. The three dimensional data of the surgical instrument 42 may beacquired in form of CAD data, a three dimensional laser scan ordimensional parameters that were entered by the user. FIG. 3 shows asingle storage medium 38 that is configured to store three dimensionalimage data for a plurality of surgical objects, such as threedimensional image data of the patient 40 and of the surgical instrument42. Alternatively, a plurality of storage media may be provided, whereinthree dimensional image data for different surgical objects is stored ondifferent storage media.

FIG. 4 shows a flow diagram of a method embodiment 60 of operating thesurgical navigation system 30 of FIG. 3 or another surgical navigationsystem.

The method 60 comprises a step 62 of determining first surface normalsfrom the three dimensional image data. The processor 36 determines, forexample, a surface of the surgical object from analysing a densitygradient of the three dimensional image data (e.g., of a CT scan) or bydetermining outer polygons of a three dimensional model of the surgicalobject. The processor 36 then reconstructs the first surface normals bycomparing the position of the corresponding area of the surface with asurrounding area of the surface. From these positions a smooth surfacemay be interpolated from which the first surface normals can becalculated. The first surface normals are assigned three-dimensionalcoordinates of corresponding surface positions in a first coordinatesystem.

According to step 64, second surface normals 20 are determined from atleast three images captured by the camera system 32 under differentpolarization angles, wherein the at least three images are captures ofthe surgical object 40, 42. To this end, the camera 34 shown in FIG. 3is configured to record three images of at least one of the patient 40and the surgical instrument 42 under different polarization angles ofthe polarization filter 26. For each pixel of the camera 34, threedifferent light intensities are recorded, respectively. According to theequations described above, for each pixel a second surface normal {rightarrow over (p)} is calculated. These second surface normals {right arrowover (p)} form a polarization normal map PN, wherein each pixel of thetwo-dimensional pixel array is assigned a three-component normal vector{right arrow over (p)}:PN:Z ² →R ³ ,PN(x′,y′)={right arrow over (p)}(x′,y′).  (Eq. 7)wherein x′ and y′ are coordinates of a second coordinate system. Thepixel array is aligned with an x′ axis and a y′ axis of a secondcoordinate system.

Alternatively, a different polarization normal map PN₂ may beconsidered, wherein the second surface normals are projected onto theimage plane 16 of the pixel array:

$\begin{matrix}{{{PN}_{2} = \left. Z^{2}\rightarrow R^{2} \right.},{{{PN}_{2}\left( {x^{\prime},y^{\prime}} \right)} = {{P \cdot {\overset{\rightarrow}{p}\left( {x^{\prime},y^{\prime}} \right)}} = {\begin{bmatrix}f_{x^{\prime}} & 0 & c_{x^{\prime}} \\0 & f_{y^{\prime}} & c_{y^{\prime}} \\0 & 0 & 1\end{bmatrix} \cdot {\overset{\rightarrow}{p}\left( {x^{\prime},y^{\prime}} \right)}}}},} & \left( {{Eq}.\mspace{11mu} 8} \right)\end{matrix}$with coordinates of a focal length f_(x′), f_(y′) of the camera 34, andcoordinates of a projection center c_(x′) and c_(y′).

The method 60 further comprises a step 66 of aligning the first surfacenormals with the second surface normals. FIG. 5 shows an alignmentprocedure of the first surface normals and the second surface normals.

The three dimensional image data 43 identifies spatial points in thefirst coordinate system 44A (with coordinates x, y and z). The firstcoordinate system 44A may, for example, be a coordinate system usedduring a CT scan or for designing a CAD drawing of a surgical object.The processor 36 is configured to assign each first surface normal acoordinate in the first coordinate system 44A. Areas of the surgicalobject other than the surface may also be assigned coordinates. Forexample, spatial points of non-exposed organs of the patient 40, whichare therefore not part of the surface captured by the camera 34, areassigned coordinates in the first coordinate system 44A.

The second coordinate system 44B (with coordinates x′, y′ and z′), onthe other hand, is arranged relative to the camera system 32. Onepossible way to arrange the second coordinate system 44B is to align itwith the view point of the camera 34. For example, the origin of thesecond coordinate system 44B may be placed in the center of the imageplane of the camera 34, wherein one of its axes (e.g., the z′-axis) isaligned with the viewing direction of the camera. One advantage of suchan orientation is that the image plane and pixel array of the camera 34extend parallel to the x′-y′-plane of the second coordinate system 44B,which allows assignment of second surface normals of the normal map PNto only the x′ and y′ axis.

The first surface normals are assigned to three dimensional data 43,wherein the second surface normals are in form of a normal map PNgenerated from the viewpoint of the camera 34. In order to align thefirst surface normals with the second surface normals, the first surfacenormals need to be transformed into a model normal map MN that allowsassignment to the polarization normal map PN of the second coordinatesystem 44B.

For this purpose, a virtual view point 50 is introduced that simulatesthe view of the camera 34 in the first coordinate system 44A. The modelnormal map MN is calculated from the virtual view point 50. Such avirtual view point has a viewing direction and a virtual image planeperpendicular thereto. Therefore, one way to implement such a virtualview point 50 is in form of an intermediate third coordinate system 44C(with coordinates x″, y″ and z″) arranged in the first coordinate system44A. The virtual viewpoint 50 is in a coordinate origin O_(C) of thethird coordinate system 44C, which is to be aligned with a secondcoordinate system 44B.

Initially, the position and orientation of the third coordinate system44C are unknown and will be determined during the alignment procedure.Since the alignment procedure requires a comparison between the twonormal maps PN and MN and therefore the calculation of a model normalmap MN, an initial position of the third coordinate system 44C isarbitrarily selected. The initial position may be, for example, theorigin of the first coordinate system 44A, a position of a previousalignment (performed during a previous surgery), an average position ofprevious alignments or a position selected by the user.

The processor 36 is configured to calculate the model normal map MN ofthe first surface normals from the virtual view point 50. The modelnormal map MN is structured like the polarization normal map NP, whereininteger coordinates x″, y″∈Z are assigned to normal vectors:MN:Z ² →R ³ ,MN(x″,y″)={right arrow over (m)}(x″,y″)  (Eq. 9)

During alignment, the second and third coordinate system 44B, 44C is tobe aligned in such a way that the x and y axis of the second and thirdcoordinate system 44B, 44C align, i.e. x′=x″ and y′=y″. The model normalmap MN may have the same size and side ratios as the polarization normalmap PN in order to allow a one-to-one assignment of vectors of bothnormal maps.

The processor 36 has access to parameters of the camera system 32 thatare indicative of a spatial position or optical properties of the camera34. Such a parameter may be a spatial relation between the camera 34 andthe surgical object 40, 42 or other surgical elements such as a surgicalbed, the floor or a reference marker. The processor 36 may alsodetermine the spatial relation relative to the surgical object 40, 42via optical properties of the camera 34, such as focal length ordistortion. Since the processor 36 is configured to calculate thespatial relation of the camera 34 relative to the surgical object, thedistance between the camera 34 and the surgical object can becalculated. Consequently, the processor 36 is configured to use the samedistance between the virtual view point 50 and the surgical object inthe three dimensional data 43 as the distance determined between thecamera 34 and the surgical object 40, 42.

The processor 36 is configured to align the first surface normals withthe second surface normals. To this end, the processor 36 is configuredto successively reposition and reorient the virtual view point 50 insidethe first coordinate system 44A in such a way that a difference betweenthe model normal map MN (determined for the corresponding virtual viewpoint 50) and the polarization normal map PN is minimized.

The difference between the normal maps may be defined as a sum ofdifferences between first surface normals and second surface normals. Inparticular, a difference d between two surface normals may be defined asd(x′,y′)=∥{right arrow over (p)}(x′,y′)−(x″,y″)∥²  (Eq. 10)

Alternatively, the difference may be calculated between projections ofthe normals onto the image plane with a projection matrix, such as theone defined in equation (8):d(x′,y′)=∥P·{right arrow over (p)}(x′,y′)−P·{right arrow over(m)}(x″,y″)∥²  (Eq. 11)

With a difference between a first and a second surface normal defined,one example of a difference D to be minimized may be a sum of suchdifferences d:

$\begin{matrix}{{D = {{\sum\limits_{W}\;{\sum\limits_{V}{d\left( {x^{\prime},y^{\prime}} \right)}}} = {\sum\limits_{W}\;{\sum\limits_{V}{{{\overset{\rightarrow}{p}\left( {x^{\prime},y^{\prime}} \right)} - {\overset{\rightarrow}{m}\left( {x^{''},y^{''}} \right)}}}^{2}}}}},} & \left( {{Eq}.\mspace{11mu} 12} \right)\end{matrix}$wherein W and V are parameters that define a set of surface normals tobe compared. The set of surface normals may be defined as a section ofthe pixel array (e.g., a rectangle), a set of normals that areassociated with a predefined colour value of the pixel (such as a skincolour), or a set of normals that fulfil a predefined polarizationcriteria (see further below). Alternatively, the entire set of surfacenormals may be selected. In this case, W and V span along the entirerows and columns of the pixel array.

The processor 36 is configured to determine, whether the model normalmap MN and the polarization normal map PN are sufficiently aligned. Anexample criterion for the alignment is a predetermined threshold for thedifference D. The processor 36 may be configured to determine that thealignment is insufficient of the difference D is above the predeterminedthreshold. The predetermined threshold may be set by the user, in orderto adjust the accuracy of the alignment. The predetermined threshold mayalso be dependent on the surface curvature of the surgical object.Alignment of a round surface (such as a head of the patient) may requirea lower threshold than alignment of a flat surface (such as a stomacharea of the patient). Other examples for alignment criteria are apredetermined amount of iteration steps and/or a minimized difference Dinside of boundary conditions. A minimized difference D inside ofboundary conditions may be defined as the lowest difference determinedfor a predetermined spatial and angular range or for a predeterminedamount of iteration steps.

If the processor 36 determines that the alignment is insufficient, theprocessor 36 is configured to determine a new position and orientation(from here on defined as “pose”) of the virtual view point 50. The newvirtual view point 50 may be determined in different ways. The processor36 may be configured to set the new virtual view point 50 by changingthe pose by a predetermined offset. Alternatively, the processor 36 maybe configured to determine the offset of the new virtual view point 50based on a difference D determined for previous virtual view points 50.If, for example, changing the position of the virtual view point 50gradually increases the difference D, the processor 36 may be configuredto change the position in an opposite direction or change theorientation of the virtual view point 50 instead. Alternatively, theprocessor 36 may be configured to determine the difference D for a firstset of multiple poses and determine a second set of poses in theproximity of a pose of the first set of poses that yields the lowestdifference D.

The processor 36 is configured to calculate a new model normal map MN atthe new pose of the virtual view point 50. If the above mentionedalignment criterion for the new virtual new point 50 is not met, a newpose of the virtual view point 50 is determined as explained above. Thecalculation of the difference D and the determining of a new pose of thevirtual view point 50 are iteratively repeated until the alignmentcriterion is met. A virtual view point 50 that meets the alignmentcriteria is defined as an aligned virtual view point 50.

It should be noted that for the purpose of the alignment, the ambiguityof the azimuth angle φ does not need to be resolved. While surfacereconstruction usually requires resolving the azimuth ambiguity,experience has been shown that the azimuth ambiguity does not have asignificant effect on the alignment procedure.

With the first and second surface normals being aligned, atransformation between the different coordinate systems may bedetermined. As explained above, the view point 50 is correlated with thethird coordinate system 44C. Once the alignment criterion is met, thethird coordinate system 44C is arranged at and oriented according to theimage plane of the aligned virtual view point 50. The processor 36 isconfigured to determine a transformation T (such as a rotation and/ortranslation matrix) that translates and rotates the first coordinatesystem 44A onto the third coordinate system 44C.

The resulting transformation is a coordinate transformation between thefirst coordinate system 44A and the third coordinate system 44C. Forexample, applying the transformation T onto the coordinates of thevirtual viewpoint 50 in the first coordinate system 44A will result inthe coordinates (0, 0, 0), because in the third coordinate system 44Cthe virtual view point is in the origin O_(C) of the third coordinatesystem 44C. Therefore,T({right arrow over (r)} _(virtualviewpoint))={right arrow over(0)}″,  (Eq. 14)wherein {right arrow over (0)}″ is the position of the origin incoordinates of the third coordinate system 44C.

The processor 36 is configured to determine in inverse transformationT⁻¹, whereinT·T ⁻¹=1.  (Eq. 15)

Consequently, the inverse transformation T⁻¹ transforms coordinates ofthe third coordinate system 44C into coordinates of the first coordinatesystem 44A. Using the same point as above, applying the inversetransformation T⁻¹ onto the origin O_(C) of the third coordinate system44C results in the coordinates of the virtual view point inside thefirst coordinate system 44A:T ⁻¹({right arrow over (0)}″)={right arrow over (r)}_(virtualviewpoint).  (Eq. 16)

The transformations T and T⁻¹ allow calculation of point positions thatare known in one coordinate system into another coordinate system.

Since the aligned virtual view point 50 has the same spatial relation tothe surgical objects 40, 42 as the camera 34, the transformations T andT⁻¹ are also coordinate transformations between the first coordinatesystem 44A and the second coordinate system 44B.

To give an example, the first coordinate system 44A contains threedimensional image data 43, such as positions of skin surfaces, bones andorgans of the patient, but is not aligned with the patient duringsurgery. In the second coordination system 44B, on the other hand, theposition and orientation of the patient during surgery is known, but itonly contains information about the surface in form of surface normals.After aligning the second and third coordination system 44B, 44C, thethree dimensional image data 43 can be transferred to the secondcoordinate system 44B via the transformation T. A CT scan may, forexample, contain the coordinates {right arrow over (r)}^(tumour) of atumour in the first coordination system 44A. The position of the tumour{right arrow over (r)}_(tumour) in the second coordination system 44B(i.e., the coordination system of the camera 34) can be calculated withequation (17):{right arrow over (r)}′ _(tumour) =T({right arrow over (r)}_(tumour))  (Eq. 17)

The processor 36 may be configured to superimpose an image of the tumourand of the images recorded with the camera 34 and send the superimposedimage to a monitor (not shown) for the surgeon to see.

The same principle applies to other surgical objects, such as a surgicalinstrument 42. It is also possible to register a plurality of surgicalobjects. As shown in FIG. 3, two surgical objects in form of a patient40 and a surgical instrument 42 are in the field of view of the camera34. Therefore, both surgical objects 40, 42 have the same secondcoordinate system 44B. However, the first coordinate systems 44A aredifferent, for example because the three dimensional image 43 data ofthe patient 40 is a CT scan, whereas the three dimensional image data 43of the surgical instrument 42 is a CAD data set.

Consequently, both surgical objects (patient 40 and surgical instrument42) need to be aligned separately, which results in two differenttransformations T_(patient) and T_(instrument). Once these twotransformations are determined, three-dimensional coordinates of bothsurgical objects 40, 42 can each be transformed into the common secondcoordinate system 44B. The processor 36 can therefore calculate aspatial relation between the patient 40 and the surgical instrument 42.The processor 36 may be configured to superimpose the three dimensionaldata 43 of the patient 40 and the surgical instrument 42 into a commonimage for presentation to the surgeon. The image may further comprisethe image recorded by the camera 34. In this way, the navigation system30 helps the surgeon navigate the surgical instrument 42 (e.g. its tip)relative to the patient 42.

The camera system 32 shown in FIG. 3 comprises a camera 34 with arotatable polarization filter 26. The rotatable polarizer 26 may berealised in different ways. It may comprise a glass or polymer sheetwith a polarizing layer, wherein the polarization angle can be adjustedby rotating the glass or polymer sheet (e.g., by means of an actuatorsuch as an electric motor or piezo-elements). Alternatively oradditionally, the polarizer may comprise optically active elements, suchas liquid crystal cells. Optically active elements can change thepolarization angle faster and more energy efficient than a mechanicallydriven polarization filter.

The camera system 32 may be configured to capture images with aplurality of cameras 34 each with its own polarization filter. Such asurgical navigation system 30 is shown in FIG. 5. The camera system 32comprises three cameras 34A, 34B, 34C, each with a polarization filters26A, 26B, 26C. The polarization filters 26A, 26B, 26C have different andfixed polarization angles and therefore allow the cameras 34A, 34B, 34Cto capture images of a same object with different polarization angles.In such an embodiment, no mechanism for rotating a polarization filteris necessary.

The aforementioned technique of alignment allows registering of asurgical object, which involves alignment of the coordinate system ofthe camera with coordinate systems of three dimensional data.

After registering the surgical object, the object may be tracked byidentifying the pose for consecutive time slots (e.g., video frames).Given enough computing capacities, the tracking can also be performedthrough the above mentioned surgical navigation (i.e., viasurface-from-polarization). It should be noted that the computationalcosts of determining the pose for each consecutive time slot is usuallylower than for the initial registration. During the registration, thealigned virtual view point 50 may need to be searched on the entiresphere around the surgical object. However, once the aligned virtualview point 50 is determined, the new virtual view point 50 for theconsecutive time slot is expected to be located close to the previousvirtual view point 50, which reduces the angular range for searching thenew virtual view point 50.

A less computationally expensive approach is to perform the registrationvia surface-from-polarization as explained above and then continuetracking through other means such as stereo photogrammetry orelectromagnetic tracking. Examples of such arrangements are shown inFIGS. 7A-7D.

FIG. 7A shows a surgical navigation system 30 with a rotatablepolarizing filter 26 in combination with a stereo camera 45 with twocameras 47A, 47B. The dashed lines indicate a camera lens behind thepolarizing filter 26.

The surgical navigation system 30 allows registration, i.e. alignment ofthe first surface normals and second surface normals of surgicalobjects. Once the surgical objects are registered, the stereo camera 45tracks movement of the surgical objects. To this end, markers may beattached to the surgical objects to be tracked by the stereo camera 45.Furthermore, a spatial relation between the camera system 32 and thestereo camera 45 is known. The processor 36 (or another processor, suchas a processor of the stereo camera) can calculate a coordinatetransformation between the camera system 32 and the stereo camera 45from the known spatial relation.

FIG. 7B shows a stereo camera 45, wherein one of the two cameras 47A,47B comprises a rotatable polarizing filter 26. The stereo camera 45also comprises other elements such as the processor and storage medium(not shown) that enable registering surgical objects as explained above.After registration, the camera 47B with the polarizing filter 26 can beused in combination with the other camera 47A in order to performtracking of the surgical objects in a stereo photogrammetry mode.Compared to the scenario of FIG. 6A, the setup requires one camera lessand has a fixed spatial relation between the cameras, which reducescalibration complexity.

FIG. 7C shows a stereo camera 45, wherein both cameras 47A, 47B comprisea rotatable polarizing filter 26. Either camera can be used forregistration. Additionally, each camera 47A, 47B can be used to registera different surgical object.

FIG. 7D shows an apparatus 46 for electromagnetic tracking incombination with a surgical navigation system 30. The apparatus 46generates an electromagnetic field that couples with surgical objects.The electromagnetic field may couple with an electrically conductivecoil that is attached to the surgical object. Through the coupling theapparatus can track the surgical object. The initial registration isperformed by the surgical navigation system 30, whereas the trackingafterwards is performed by the apparatus 46 for electromagnetictracking.

Since the surgical navigation system 30 is capable of determining adegree of polarization, further information can be gained from thecaptured images. The degree of polarization is indicative of thereflecting material of the surface. Reflective materials such as metalsor liquid surfaces tend to reflect the light specularly, which resultsin a higher degree of polarization. On the other hand, when a surfacereflects predominantly diffusively, the degree of polarization iscomparably lower. These different reflective behaviours enabledistinguishing different types of surfaces.

As explained above, for each pixel of the camera 34, a (second) surfacenormal is determined, which comprises determining the degree ofpolarization. Therefore, for each pixel of the camera 34, the degree ofpolarization can be recorded. The processor 36 may be configured todetermine if a surgical object is of a first type if a degree ofpolarization determined for a second surface normal is lower than afirst threshold. The first type surgical object may be the patient or asurgical instrument with a non-metallic casing. Such surgical objectsreflect light predominately diffusively, which results in a low degreeof polarization. When the processor 36 determines that the degree ofpolarization is below the first threshold, it identifies the surgicalobject as the first type (e.g., the patient).

The processor 36 may further be configured to determine that thesurgical object is of a second type if a degree of polarizationdetermined for the second surface normal is higher than a secondthreshold. The second type surgical object may be a surgical instrument(e.g., with a metal casing). Such a surgical instrument reflectspredominately specularly and therefore with a high degree ofpolarization. The processor 36 is configured to detect that thereflected light is above the second threshold and determines that thecorresponding surface normal can be assigned to the second type surgicalinstrument. The first and the second threshold may be identical.

Similarly, the processor 36 may be configured to determine whether asurface is covered by a liquid or not. During surgery, exposed surgicalobjects such as bones may be covered by liquids such as water, blood ordisinfectants. Such surfaces reflect light predominantly specularly,which results in a high degree of polarization. Therefore, the processor36 may determine if a surface is covered by a liquid, when the degree ofpolarization is above a pre-determined threshold. The processor 36 mayhave a pre-determined threshold for different types of liquid.

The degree of polarization can also indicate whether a surface reflectsor emits light. A light source usually emits unpolarized light, whereinreflected light tends to be at least partially polarized. Therefore, theprocessor 36 may be configured to determine whether a surface emitslight if a degree of polarization is below a pre-determined threshold.Distinguishing between light emission and light reflection can be usefulto distinguish between a tracker that operates with an active lightsource from a reflecting surface (e.g., a bone covered with blood). Forregistering the tracker, the processor 36 may ignore light spots with adegree of polarization above the threshold, which allows the processor36 to isolate light sources of the tracker in the captured image.

The features described in relation to the exemplary embodiments shown inthe drawings can be readily combined to result in different embodiments.It is apparent, therefore, that the present disclosure may be varied inmany ways. Such variations are not to be regarded as a departure fromthe scope of the invention as defined by the claims appended hereto.

The foregoing description is merely illustrative in nature and is in noway intended to limit the disclosure, its application, or uses. Thebroad teachings of the disclosure can be implemented in a variety offorms. Therefore, while this disclosure includes particular examples,the true scope of the disclosure should not be so limited since othermodifications will become apparent upon a study of the drawings, thespecification, and the following claims. It should be understood thatone or more steps within a method may be executed in different order (orconcurrently) without altering the principles of the present disclosure.Further, although each of the embodiments is described above as havingcertain features, any one or more of those features described withrespect to any embodiment of the disclosure can be implemented in and/orcombined with features of any of the other embodiments, even if thatcombination is not explicitly described. In other words, the describedembodiments are not mutually exclusive, and permutations of one or moreembodiments with one another remain within the scope of this disclosure.

In this application, including the definitions below, the termsprocessor, module, and controller may be replaced with the term circuit.The terms processor, module, controller, and circuit may refer to, bepart of, or include: an Application Specific Integrated Circuit (ASIC);a digital, analog, or mixed analog/digital discrete circuit; a digital,analog, or mixed analog/digital integrated circuit; a combinationallogic circuit; a field programmable gate array (FPGA); a processorcircuit (shared, dedicated, or group) that executes code; a memorycircuit (shared, dedicated, or group) that stores code executed by theprocessor circuit; other suitable hardware components that provide thedescribed functionality; or a combination of some or all of the above,such as in a system-on-chip.

The term computer-readable medium, as used herein, does not encompasstransitory electrical or electromagnetic signals propagating through amedium (such as on a carrier wave); the term computer-readable mediummay therefore be considered tangible and non-transitory. Non-limitingexamples of a non-transitory computer-readable medium are nonvolatilememory circuits (such as a flash drive, a flash memory circuit, anerasable programmable read-only memory circuit, or a mask read-onlymemory circuit), volatile memory circuits (such as a static randomaccess memory circuit or a dynamic random access memory circuit),magnetic storage media (such as an analog or digital magnetic tape or ahard disk drive), and optical storage media (such as a CD, a DVD, or aBlu-ray Disc).

The apparatuses and methods described in this application may bepartially or fully implemented by a special purpose computer created byconfiguring a general purpose computer to execute one or more particularfunctions embodied in computer programs. The functional blocks and/orflow diagrams elements described above may serve as softwarespecifications, which can be translated into the computer applicationsor programs by the routine work of a skilled technician or programmer.

The computer applications or programs include processor-executableinstructions that are stored on at least one non-transitorycomputer-readable medium. The computer application or programs may alsoinclude or rely on stored data. The computer application or programs mayencompass a basic input/output system (BIOS) that interacts withhardware of the special purpose computer, device drivers that interactwith particular devices of the special purpose computer, one or moreoperating systems, user applications, background services, backgroundapplications, etc.

The computer applications or programs may, for example purposes only,include: (i) descriptive text to be parsed, such as HTML (hypertextmarkup language), XML (extensible markup language), or JSON (JavaScriptObject Notation), (ii) assembly code, (iii) object code generated fromsource code by a compiler, (iv) source code for execution by aninterpreter, (v) source code for compilation and execution by ajust-in-time compiler, etc. The source code may be written using syntaxfrom any suitable language.

What is claimed is:
 1. A surgical navigation system comprising a camera system comprising at least one camera, wherein the camera system is capable of capturing images of light that is filtered through at least three different polarization angles; a processor having access to three dimensional image data of a surgical object and being configured to: determine first surface normals from the three dimensional image data of the surgical object; determine second surface normals of the surgical object from at least three images captured of the surgical object by the camera system under different polarization angles; and align the first surface normals with the second surface normals.
 2. The surgical navigation system according to claim 1, wherein the processor is configured to align the second surface normals with the first surface normals to determine a transformation of coordinates of the first surface normals in a first coordinate system into a second coordinate system of the second surface normals.
 3. The surgical navigation system according to claim 2, wherein the transformation is determined by a minimization of a sum of differences between the first surface normals and the second surface normals.
 4. The surgical navigation system according to claim 3, wherein the processor is configured to calculate a difference between two surface normals as the magnitude of a vector difference between the two surface normals.
 5. The surgical navigation system according to claim 2, wherein the transformation is determined by a minimization of a sum of differences between projections of the first surface normals and the second surface normals onto an image plane of the camera system.
 6. The surgical navigation system according to claim 1, wherein the processor is configured to differentiate if the surgical object is of a first type or a second type based on a degree of polarization determined for the second surface normals of the surgical object.
 7. The surgical navigation system according to claim 6, wherein the processor is configured to at least one of determine that the surgical object is of the first type if a degree of polarization determined for the second surface normals of the surgical object is lower than a first threshold and to determine that the surgical object is of the second type if a degree of polarization determined for the second surface normals of the surgical object is higher than a second threshold.
 8. The surgical navigation system according to claim 1, wherein the processor is configured to differentiate if a second surface normal of the second surface normals is pertaining to a surface or surface region covered by liquid based on a degree of polarization determined for the second surface normal of the second surface normals.
 9. The surgical navigation system according to claim 8, wherein the processor is configured to determine that the second surface normal of the second surface normals is pertaining to a surface or surface region covered by liquid if the degree of polarization determined for the second surface normal of the second surface normals is above a third threshold.
 10. The surgical navigation system according to claim 1, wherein the processor is configured to differentiate if a second surface normal of the second surface normals is pertaining to a surface emitting or reflecting light based on a degree of polarization determined for the second surface normal of the second surface normals.
 11. The surgical navigation system according to claim 10, wherein the processor is configured to at least one of determine that the second surface normal of the second surface normals is pertaining to a light emitting surface if the degree of polarization determined for the second surface normal of the second normals is below a fourth threshold and to determine that the second surface normal of the second surface normals is pertaining to a light reflecting surface if the degree of polarization determined for the second surface normal of the second normals is above a fifth threshold.
 12. The surgical navigation system according to claim 1, wherein the processor is configured to determine the second surface normals from the at least three images by fitting a light intensity of pixels of the captured images to a sinusoid model that describes a magnitude of a light beam having a light beam polarization angle passing through a polarization filter.
 13. The surgical navigation system according to claim 1, wherein the processor is configured to determine the second surface normals from the at least three images by using a Fresnel equation.
 14. The surgical navigation system according to claim 1, wherein the processor is configured to determine or have access to at least one parameter of at least one camera of the camera system that is indicative of at least one of a spatial position and optical properties of the at least one camera of the camera system.
 15. The surgical navigation system according to claim 1, wherein the processor is further configured to track a second type surgical object relative to a first type surgical object by calculating a position and orientation of the second type surgical object relative to the first type surgical object.
 16. The surgical navigation system according to claim 15, wherein the first type surgical object is a patient or a part thereof and the second type surgical object is a surgical instrument.
 17. The surgical navigation system according to claim 1, wherein the three dimensional data was previously acquired from at least one of an unknown position and an unknown viewing angle.
 18. The surgical navigation system according to claim 1, wherein the camera system comprises optically active elements configured to rotate a polarization angle of polarized light entering the camera system.
 19. A method for operating a surgical navigation system, the surgical navigation system comprising a camera system comprising at least one camera, wherein the camera system is capable of capturing images of light that is filtered through at least three different polarization angles, and having access to three dimensional image data of a surgical object; the method comprising: determining first surface normals of the surgical object from the three dimensional image data; determining second surface normals of the surgical object from at least three images captured by the camera system of the surgical object under different polarization angles; and aligning the first surface normals with the second surface normals.
 20. A non-transitory computer-readable medium that stores programming instructions for execution, that when executed on at least one processor, cause the at least one processor to carry out a method comprising: determining first surface normals of a surgical object from three dimensional image data; determining second surface normals of a surgical object from at least three images captured by a camera system of the surgical object under different polarization angles; and aligning the first surface normals with the second surface normals. 